Fitness training system with energy expenditure calculation that uses a form factor

ABSTRACT

System and methods are provided for prompting a user to perform an exercise and to monitor the exercise. The form of the user may be monitored, such as with one or more video cameras and/or other sensors, to determine how well the user is performing the exercise. Energy expenditure is estimated based factors that include the type of the exercise, the form of the user and a contribution value that associates energy expenditure with form.

The present application is a continuation-in-part of U.S. patentapplication Ser. No. 13/324,812, filed Dec. 13, 2011, entitled“Processing Data of a User Performing an Athletic Activity to EstimateEnergy Expenditure,” which is a continuation-in-part of U.S. patentapplication Ser. No. 13/304,056, filed Nov. 23, 2011, entitled “FatigueIndices and Uses Thereof,” and of U.S. patent application Ser. No.13/304,064, filed Nov. 23, 2011, entitled “Method and System forAutomated Personal Training that Includes Training Programs,” which is acontinuation-in-part of U.S. application Ser. No. 13/290,359, filed Nov.7, 2011, entitled “Method and System for Automated Personal Training,”which claims the benefit of and priority to, U.S. Provisional PatentApplication Ser. No. 61/433,792 filed Jan. 18, 2011, 61/432,472 filedJan. 13, 2011, and 61/422,511 filed Dec. 13, 2010, each of which isentitled “Method and System for Automated Personal Training.” Thepresent application also claims priority to U.S. Provisional PatentApplication Ser. No. 61/655,153, filed Jun. 4, 2012, entitled “FitnessTraining System with Energy Expenditure Calculation that Uses a FormFactor.” The contents of each of the above-identified applications areexpressly incorporated herein by reference in its entirety for any andall non-limiting purposes.

BACKGROUND

While most people appreciate the importance of physical fitness, manyhave difficulty finding the motivation required to maintain a regularexercise program. Some people find it particularly difficult to maintainan exercise regimen that involves continuously repetitive motions, suchas running, walking and bicycling.

Additionally, individuals may view exercise as work or a chore and thus,separate it from enjoyable aspects of their daily lives. Often, thisclear separation between athletic activity and other activities reducesthe amount of motivation that an individual might have towardexercising. Further, athletic activity services and systems directedtoward encouraging individuals to engage in athletic activities mightalso be too focused on one or more particular activities while anindividual's interest are ignored. This may further decrease a user'sinterest in participating in athletic activities or using the athleticactivity services and systems.

Therefore, improved systems and methods to address these and othershortcomings in the art are desired.

BRIEF SUMMARY

The following presents a simplified summary in order to provide a basicunderstanding of some aspects of the disclosure. The summary is not anextensive overview of the disclosure. It is neither intended to identifykey or critical elements of the disclosure nor to delineate the scope ofthe disclosure. The following summary merely presents some concepts ofthe disclosure in a simplified form as a prelude to the descriptionbelow.

Aspects of this disclosure relate to processing of data taken while auser performs an athletic activity to determine an estimate of energyexpenditure such as, for example, an amount of calories burned.

Example embodiments may relate to a system, method, apparatus, andcomputer readable media configured for prompting a user to perform anexercise, monitoring form of the user while performing the exercise, andcalculating an energy expenditure estimate for the user performing theexercise based on a type of the exercise and on the form of the user. Inother embodiments, expenditure estimate may be, or comprise, forexample, an estimate of calories burned by the user. In certainembodiments, energy expenditure calculations comprise determinationsrelating to: effort, oxygen consumed, and/or oxygen kinetics of theuser.

In various aspects, a system, method, apparatus, and/or computerreadable media may be configured for processing data captured of a userperforming an athletic activity over a time interval, and determining alocation of a center of mass of a body part, body region, or entire bodyof the user at a first time instant and at a second time instant withinthe time interval. In further aspects, a system, method, apparatus,and/or computer readable media may be configured for identifying achange in the location of the center of mass from the first time instantto the second time instant, and calculating an energy expenditureestimate for the user due to the change.

These and other aspects of the embodiments are discussed in greaterdetail throughout this disclosure, including the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example and not limitedin the accompanying figures in which like reference numerals indicatesimilar elements and in which:

FIGS. 1A-B illustrate an example of a system for providing personaltraining in accordance with example embodiments, wherein FIG. 1Aillustrates an example network configured to monitor athletic activity,and FIG. 1B illustrates an example computing device in accordance withexample embodiments.

FIGS. 2A-B illustrate example sensor assemblies that may be worn by auser in accordance with example embodiments.

FIG. 3 illustrates an example flow diagram of a method for calculatingan energy expenditure estimate for a user that accounts for a user'sform while exercising as part of the estimate, in accordance withexample embodiments.

FIG. 4 illustrates example points on a user's body for monitoring duringexercising in accordance with example embodiments.

FIG. 5 illustrates an example posture assessment in accordance withexample embodiments.

FIG. 6 illustrates example displays of a virtual avatar of a userperforming an exercise in accordance with example embodiments.

FIGS. 7A-B illustrate example displays of a virtual avatar of a userperforming a squat in accordance with example embodiments.

FIG. 8 illustrates an example flow diagram of a method for calculatingan energy expenditure estimate for a user while performing an athleticactivity based on monitoring changes in potential energy, in accordancewith example embodiments.

FIGS. 9, 10A-B, and 11 illustrate example locations of centers of massfor a virtual avatar of user, in accordance with example embodiments.

DETAILED DESCRIPTION

In the following description of the various embodiments, reference ismade to the accompanying drawings, which form a part hereof, and inwhich is shown by way of illustration various embodiments in which thedisclosure may be practiced. It is to be understood that otherembodiments may be utilized and structural and functional modificationsmay be made without departing from the scope and spirit of the presentdisclosure. Further, headings within this disclosure should not beconsidered as limiting aspects of the disclosure. Those skilled in theart with the benefit of this disclosure will appreciate that the exampleembodiments are not limited to the example headings.

I. Example Personal Training System A. Illustrative Computing Devices

FIG. 1A illustrates an example of a personal training system 100 inaccordance with example embodiments. Example system 100 may include oneor more electronic devices, such as computer 102. Computer 102 maycomprise a mobile terminal, such as a telephone, music player, tablet,netbook or any portable device. In other embodiments, computer 102 maycomprise a set-top box (STB), desktop computer, digital videorecorder(s) (DVR), computer server(s), and/or any other desiredcomputing device. In certain configurations, computer 102 may comprise agaming console, such as for example, a Microsoft® XBOX, Sony®Playstation, and/or a Nintendo® Wii gaming consoles. Those skilled inthe art will appreciate that these are merely example consoles fordescriptive purposes and this disclosure is not limited to any consoleor device.

Turning briefly to FIG. 1B, computer 102 may include computing unit 104,which may comprise at least one processing unit 106. Processing unit 106may be any type of processing device for executing softwareinstructions, such as for example, a microprocessor device. Computer 102may include a variety of non-transitory computer readable media, such asmemory 108. Memory 108 may include, but is not limited to, random accessmemory (RAM) such as RAM 110, and/or read only memory (ROM), such as ROM112. Memory 108 may include any of: electronically erasable programmableread only memory (EEPROM), flash memory or other memory technology,CD-ROM, digital versatile disks (DVD) or other optical disk storage,magnetic storage devices, or any other medium that can be used to storethe desired information and that can be accessed by computer 102.

The processing unit 106 and the system memory 108 may be connected,either directly or indirectly, through a bus 114 or alternatecommunication structure to one or more peripheral devices. For example,the processing unit 106 or the system memory 108 may be directly orindirectly connected to additional memory storage, such as a hard diskdrive 116, a removable magnetic disk drive, an optical disk drive 118,and a flash memory card. The processing unit 106 and the system memory108 also may be directly or indirectly connected to one or more inputdevices 120 and one or more output devices 122. The output devices 122may include, for example, a display device 136, television, printer,stereo, or speakers. In some embodiments one or more display devices maybe incorporated into eyewear. The display devices incorporated intoeyewear may provide feedback to users. Eyewear incorporating one or moredisplay devices also provides for a portable display system. The inputdevices 120 may include, for example, a keyboard, touch screen, a remotecontrol pad, a pointing device (such as a mouse, touchpad, stylus,trackball, or joystick), a scanner, a camera or a microphone. In thisregard, input devices 120 may comprise one or more sensors configured tosense, detect, and/or measure athletic movement from a user, such asuser 124, shown in FIG. 1A.

Looking again to FIG. 1A, image-capturing device 126 and/or sensor 128may be utilized in detecting and/or measuring athletic movements of user124. In one embodiment, data obtained from image-capturing device 126 orsensor 128 may directly detect athletic movements, such that the dataobtained from image-capturing device 126 or sensor 128 is directlycorrelated to a motion parameter. For example, and with reference toFIG. 4, image data from image-capturing device 126 may detect that thedistance between sensor locations 402 g and 402 i has decreased andtherefore, image-capturing device 126 alone may be configured to detectthat user's 124 right arm has moved. Yet, in other embodiments, datafrom image-capturing device 126 and/or sensor 128 may be utilized incombination, either with each other or with other sensors to detectand/or measure movements. Thus, certain measurements may be determinedfrom combining data obtained from two or more devices. Image-capturingdevice 126 and/or sensor 128 may include or be operatively connected toone or more sensors, including but not limited to: an accelerometer, agyroscope, a location-determining device (e.g., GPS), light sensor,temperature sensor (including ambient temperature and/or bodytemperature), heart rate monitor, image-capturing sensor, moisturesensor and/or combinations thereof. Example uses of illustrative sensors126, 128 are provided below in Section I.C, entitled “IllustrativeSensors.” Computer 102 may also use touch screens or image capturingdevice to determine where a user is pointing to make selections from agraphical user interface. One or more embodiments may utilize one ormore wired and/or wireless technologies, alone or in combination,wherein examples of wireless technologies include Bluetooth®technologies, Bluetooth® low energy technologies, and/or ANTtechnologies.

B. Illustrative Network

Still further, computer 102, computing unit 104, and/or any otherelectronic devices may be directly or indirectly connected to one ormore network interfaces, such as example interface 130 (shown in FIG.1B) for communicating with a network, such as network 132. In theexample of FIG. 1B, network interface 130, may comprise a networkadapter or network interface card (NIC) configured to translate data andcontrol signals from the computing unit 104 into network messagesaccording to one or more communication protocols, such as theTransmission Control Protocol (TCP), the Internet Protocol (IP), and theUser Datagram Protocol (UDP). These protocols are well known in the art,and thus will not be discussed here in more detail. An interface 130 mayemploy any suitable connection agent for connecting to a network,including, for example, a wireless transceiver, a power line adapter, amodem, or an Ethernet connection. Network 132, however, may be any oneor more information distribution network(s), of any type(s) ortopology(s), alone or in combination(s), such as internet(s),intranet(s), cloud(s), LAN(s). Network 132 may be any one or more ofcable, fiber, satellite, telephone, cellular, wireless, etc. Networksare well known in the art, and thus will not be discussed here in moredetail. Network 132 may be variously configured such as having one ormore wired or wireless communication channels to connect one or morelocations (e.g., schools, businesses, homes, consumer dwellings, networkresources, etc.), to one or more remote servers 134, or to othercomputers, such as similar or identical to computer 102. Indeed, system100 may include more than one instance of each component (e.g., morethan one computer 102, more than one display 136, etc.).

Regardless of whether computer 102 or other electronic device withinnetwork 132 is portable or at a fixed location, it should be appreciatedthat, in addition to the input, output and storage peripheral devicesspecifically listed above, the computing device may be connected, suchas either directly, or through network 132 to a variety of otherperipheral devices, including some that may perform input, output andstorage functions, or some combination thereof. In certain embodiments,a single device may integrate one or more components shown in FIG. 1A.For example, a single device may include computer 102, image-capturingdevice 126, sensor 128, display 136 and/or additional components. In oneembodiment, sensor device 138 may comprise a mobile terminal having adisplay 136, image-capturing device 126, and one or more sensors 128.Yet, in another embodiment, image-capturing device 126, and/or sensor128 may be peripherals configured to be operatively connected to a mediadevice, including for example, a gaming or media system. Thus, it goesfrom the foregoing that this disclosure is not limited to stationarysystems and methods. Rather, certain embodiments may be carried out by auser 124 in almost any location.

C. Illustrative Sensors

Computer 102 and/or other devices may comprise one or more sensors 126,128 configured to detect and/or monitor at least one fitness parameterof a user 124. Sensors 126 and/or 128 may include, but are not limitedto: an accelerometer, a gyroscope, a location-determining device (e.g.,GPS), light sensor, temperature sensor (including ambient temperatureand/or body temperature), sleep pattern sensors, heart rate monitor,image-capturing sensor, moisture sensor and/or combinations thereof.Network 132 and/or computer 102 may be in communication with one or moreelectronic devices of system 100, including for example, display 136, animage capturing device 126 (e.g., one or more video cameras), and sensor128, which may be an infrared (IR) device. In one embodiment sensor 128may comprise an IR transceiver. For example, sensors 126, and/or 128 maytransmit waveforms into the environment, including towards the directionof user 124 and receive a “reflection” or otherwise detect alterationsof those released waveforms. In yet another embodiment, image-capturingdevice 126 and/or sensor 128 may be configured to transmit and/orreceive other wireless signals, such as radar, sonar, and/or audibleinformation. Those skilled in the art will readily appreciate thatsignals corresponding to a multitude of different data spectrums may beutilized in accordance with various embodiments. In this regard, sensors126 and/or 128 may detect waveforms emitted from external sources (e.g.,not system 100). For example, sensors 126 and/or 128 may detect heatbeing emitted from user 124 and/or the surrounding environment. Thus,image-capturing device 126 and/or sensor 128 may comprise one or morethermal imaging devices. In one embodiment, image-capturing device 126and/or sensor 128 may comprise an IR device configured to perform rangephenomenology. As a non-limited example, image-capturing devicesconfigured to perform range phenomenology are commercially availablefrom Flir Systems, Inc. of Portland, Oreg. Although image capturingdevice 126 and sensor 128 and display 136 are shown in direct(wirelessly or wired) communication with computer 102, those skilled inthe art will appreciate that any may directly communicate (wirelessly orwired) with network 132.

1. Multi-Purpose Electronic Devices

User 124 may possess, carry, and/or wear any number of electronicdevices, including sensory devices 138, 140, 142, and/or 144. In certainembodiments, one or more devices 138, 140, 142, 144 may not be speciallymanufactured for fitness or athletic purposes. Indeed, aspects of thisdisclosure relate to utilizing data from a plurality of devices, some ofwhich are not fitness devices, to collect, detect, and/or measureathletic data. In one embodiment, device 138 may comprise a portableelectronic device, such as a telephone or digital music player,including an IPOD®, IPAD®, or iPhone®, brand devices available fromApple, Inc. of Cupertino, Calif. or Zune® or Microsoft® Windows devicesavailable from Microsoft of Redmond, Wash. As known in the art, digitalmedia players can serve as both an output device for a computer (e.g.,outputting music from a sound file or pictures from an image file) and astorage device. In one embodiment, device 138 may be computer 102, yetin other embodiments, computer 102 may be entirely distinct from device138. Regardless of whether device 138 is configured to provide certainoutput, it may serve as an input device for receiving sensoryinformation. Devices 138, 140, 142, and/or 144 may include one or moresensors, including but not limited to: an accelerometer, a gyroscope, alocation-determining device (e.g., GPS), light sensor, temperaturesensor (including ambient temperature and/or body temperature), heartrate monitor, image-capturing sensor, moisture sensor and/orcombinations thereof. In certain embodiments, sensors may be passive,such as reflective materials that may be detected by image-capturingdevice 126 and/or sensor 128 (among others). In certain embodiments,sensors 144 may be integrated into apparel, such as athletic clothing.For instance, the user 124 may wear one or more on-body sensors 144 a-b.Sensors 144 may be incorporated into the clothing of user 124 and/orplaced at any desired location of the body of user 124. Sensors 144 maycommunicate (e.g., wirelessly) with computer 102, sensors 128, 138, 140,and 142, and/or camera 126. Examples of interactive gaming apparel aredescribed in U.S. patent application Ser. No. 10/286,396, filed Oct. 30,2002, and published as U.S. Pat. Pub, No. 2004/0087366, the contents ofwhich are incorporated herein by reference in its entirety for any andall non-limiting purposes. In certain embodiments, passive sensingsurfaces may reflect waveforms, such as infrared light, emitted byimage-capturing device 126 and/or sensor 128. In one embodiment, passivesensors located on user's 124 apparel may comprise generally sphericalstructures made of glass or other transparent or translucent surfaceswhich may reflect waveforms. Different classes of apparel may beutilized in which a given class of apparel has specific sensorsconfigured to be located proximate to a specific portion of the user's124 body when properly worn. For example, golf apparel may include oneor more sensors positioned on the apparel in a first configuration andyet soccer apparel may include one or more sensors positioned on apparelin a second configuration.

Devices 138-144 may communicate with each other, either directly orthrough a network, such as network 132. Communication between one ormore of devices 138-144 may communicate through computer 102. Forexample, two or more of devices 138-144 may be peripherals operativelyconnected to bus 114 of computer 102. In yet another embodiment, a firstdevice, such as device 138 may communicate with a first computer, suchas computer 102 as well as another device, such as device 142, however,device 142 may not be configured to connect to computer 102 but maycommunicate with device 138. Those skilled in the art will appreciatethat other configurations are possible.

Some implementations of the example embodiments may alternately oradditionally employ computing devices that are intended to be capable ofa wide variety of functions, such as a desktop or laptop personalcomputer. These computing devices may have any combination of peripheraldevices or additional components as desired. Also, the components shownin FIG. 1B may be included in the server 134, other computers,apparatuses, etc.

2. Illustrative Apparel/Accessory Sensors

In certain embodiments, sensory devices 138, 140, 142 and/or 144 may beformed within or otherwise associated with user's 124 clothing oraccessories, including a watch, armband, wristband, necklace, shirt,shoe, or the like. Examples of shoe-mounted and wrist-worn devices(devices 140 and 142, respectively) are described immediately below,however, these are merely example embodiments and this disclosure shouldnot be limited to such.

i. Shoe-Mounted Device

In certain embodiments, sensory device 140 may comprise footwear whichmay include one or more sensors, including but not limited to: anaccelerometer, location-sensing components, such as GPS, and/or a forcesensor system. FIG. 2A illustrates one example embodiment of a sensorsystem 202. In certain embodiments, system 202 may include a sensorassembly 204. Assembly 204 may comprise one or more sensors, such as forexample, an accelerometer, location-determining components, and/or forcesensors. In the illustrated embodiment, assembly 204 incorporates aplurality of sensors, which may include force-sensitive resistor (FSR)sensors 206. In yet other embodiments, other sensor(s) may be utilized.Port 208 may be positioned within a sole structure 209 of a shoe. Port208 may optionally be provided to be in communication with an electronicmodule 210 (which may be in a housing 211) and a plurality of leads 212connecting the FSR sensors 206 to the port 208. Module 210 may becontained within a well or cavity in a sole structure of a shoe. Theport 208 and the module 210 include complementary interfaces 214, 216for connection and communication.

In certain embodiments, at least one force-sensitive resistor 206 shownin FIG. 2A may contain first and second electrodes or electricalcontacts 218, 220 and a force-sensitive resistive material 222 disposedbetween the electrodes 218, 220 to electrically connect the electrodes218, 220 together. When pressure is applied to the force-sensitivematerial 222, the resistivity and/or conductivity of the force-sensitivematerial 222 changes, which changes the electrical potential between theelectrodes 218, 220. The change in resistance can be detected by thesensor system 202 to detect the force applied on the sensor 216. Theforce-sensitive resistive material 222 may change its resistance underpressure in a variety of ways. For example, the force-sensitive material222 may have an internal resistance that decreases when the material iscompressed, similar to the quantum tunneling composites described ingreater detail below. Further compression of this material may furtherdecrease the resistance, allowing quantitative measurements, as well asbinary (on/off) measurements. In some circumstances, this type offorce-sensitive resistive behavior may be described as “volume-basedresistance,” and materials exhibiting this behavior may be referred toas “smart materials.” As another example, the material 222 may changethe resistance by changing the degree of surface-to-surface contact.This can be achieved in several ways, such as by using microprojectionson the surface that raise the surface resistance in an uncompressedcondition, where the surface resistance decreases when themicroprojections are compressed, or by using a flexible electrode thatcan be deformed to create increased surface-to-surface contact withanother electrode. This surface resistance may be the resistance betweenthe material 222 and the electrode 218, 220 222 and/or the surfaceresistance between a conducting layer (e.g., carbon/graphite) and aforce-sensitive layer (e.g., a semiconductor) of a multi-layer material222. The greater the compression, the greater the surface-to-surfacecontact, resulting in lower resistance and enabling quantitativemeasurement. In some circumstances, this type of force-sensitiveresistive behavior may be described as “contact-based resistance.” It isunderstood that the force-sensitive resistive material 222, as definedherein, may be or include a doped or non-doped semiconducting material.

The electrodes 218, 220 of the FSR sensor 216 can be formed of anyconductive material, including metals, carbon/graphite fibers orcomposites, other conductive composites, conductive polymers or polymerscontaining a conductive material, conductive ceramics, dopedsemiconductors, or any other conductive material. The leads 212 can beconnected to the electrodes 218, 220 by any suitable method, includingwelding, soldering, brazing, adhesively joining, fasteners, or any otherintegral or non-integral joining method. Alternately, the electrode 218,220 and associated lead 212 may be formed of a single piece of the samematerial.

ii. Wrist-Worn Device

As shown in FIG. 2B, device 226 (which may resemble or be sensory device142 shown in FIG. 1A) may be configured to be worn by user 124, such asaround a wrist, arm, ankle or the like. Device 226 may monitor athleticmovements of a user, including all-day activity of user 124. In thisregard, device assembly 226 may detect athletic movement during user's124 interactions with computer 102 and/or operate independently ofcomputer 102. For example, in one embodiment, device 226 may be an-allday activity monitor that measures activity regardless of the user'sproximity or interactions with computer 102. Device 226 may communicatedirectly with network 132 and/or other devices, such as devices 138and/or 140. In other embodiments, athletic data obtained from device 226may be utilized in determinations conducted by computer 102, such asdeterminations relating to which exercise programs are presented to user124. In one embodiment, device 226 may also wirelessly interact with amobile device, such as device 138 associated with user 124 or a remotewebsite such as a site dedicated to fitness or health related subjectmatter. At some predetermined time, the user may wish to transfer datafrom the device 226 to another location.

As shown in FIG. 2B, device 226 may include an input mechanism, such asa depressible input button 228 assist in operation of the device 226.The input button 228 may be operably connected to a controller 230and/or any other electronic components, such as one or more of theelements discussed in relation to computer 102 shown in FIG. 1B.Controller 230 may be embedded or otherwise part of housing 232. Housing232 may be formed of one or more materials, including elastomericcomponents and comprise one or more displays, such as display 234. Thedisplay may be considered an illuminable portion of the device 226. Thedisplay 234 may include a series of individual lighting elements orlight members such as LED lights 234 in an exemplary embodiment. The LEDlights may be formed in an array and operably connected to thecontroller 230. Device 226 may include an indicator system 236, whichmay also be considered a portion or component of the overall display234. It is understood that the indicator system 236 can operate andilluminate in conjunction with the display 234 (which may have pixelmember 235) or completely separate from the display 234. The indicatorsystem 236 may also include a plurality of additional lighting elementsor light members 238, which may also take the form of LED lights in anexemplary embodiment. In certain embodiments, indicator system mayprovide a visual indication of goals, such as by illuminating a portionof lighting members 238 to represent accomplishment towards one or moregoals.

A fastening mechanism 240 can be unlatched wherein the device 226 can bepositioned around a wrist of the user 124 and the fastening mechanism240 can be subsequently placed in a latched position. The user can wearthe device 226 at all times if desired. In one embodiment, fasteningmechanism 240 may comprise an interface, including but not limited to aUSB port, for operative interaction with computer 102 and/or devices138, 140.

In certain embodiments, device 226 may comprise a sensor assembly (notshown in FIG. 2B). The sensor assembly may comprise a plurality ofdifferent sensors. In an example embodiment, the sensor assembly maycomprise or permit operative connection to an accelerometer (includingin the form of a multi-axis accelerometer), heart rate sensor,location-determining sensor, such as a GPS sensor, and/or other sensors.Detected movements or parameters from device's 142 sensor(s), mayinclude (or be used to form) a variety of different parameters, metricsor physiological characteristics including but not limited to speed,distance, steps taken, and energy expenditure such as calories, heartrate, sweat detection, effort, oxygen consumed, and/or oxygen kinetics.Such parameters may also be expressed in terms of activity points orcurrency earned by the user based on the activity of the user.

Illustrative Athletic Monitoring Methods

System 100 may prompt a user to perform one or more exercises, monitoruser movement while performing the exercises, and provide the user withan energy expenditure estimate based on their movement. System 100 mayanalyze a user's form to determine if the user is making an exercisemore or less difficult, and adjust the energy expenditure estimateaccordingly. Energy expenditure estimates may be, or comprise, anestimate of calories burned by the user. In certain embodiments, energyexpenditure determinations may be based on, and/or conveyed as a pointsystem. In one embodiment, calories may be converted to a point system,yet in other embodiments, measurements may be directly obtained in oneor more point systems. In one implementation, activity points may bebased upon: form, body movements, and/or completion of certainactivities. In further embodiments, energy expenditure calculations maycomprise determinations relating to: effort, oxygen consumed, and/oroxygen kinetics of the user. In one embodiment, computer 102, camera126, sensor 128, and display 136 may be implemented within the confinesof a user's residence, although other locations, including gyms and/orbusinesses are contemplated. Further, as discussed above, computer 102may be a portable device, such as a cellular telephone, therefore, oneor more aspects discussed herein may be conducted in almost anylocation. In this regard, the example embodiments of this disclosure arediscussed in the context of being implemented with one or more of theexample components of system 100. Those skilled in the art willappreciate that reference(s) to a particular component, such as computer102, is not meant to be limiting, but rather to provide an illustrativeexample of one of many possible implementations. Thus, although certaincomponents may be referenced, it is to be assumed that other componentsof system 100 may be utilized unless expressly disclaimed or physicallyimpossible. Further, aspects disclosed herein are not limited to examplesystem 100.

A. Monitoring User Movements

While exercising, the system 100 may use one or more techniques tomonitor user movement. FIG. 3 illustrates an example flow diagram of amethod for calculating an energy expenditure estimate for a user thataccounts for a user's form while exercising as part of the estimate, inaccordance with example embodiments. The method may be implemented by acomputer, such as, for example, computer 102, device 138, 140 and/or142, as well as or other apparatuses. The blocks shown in FIG. 3 may berearranged, some blocks may be removed, additional blocks may be added,each block may be repeated one or more times, and the flow diagram maybe repeated one or more times. The flow diagram may begin at block 302.

1. Perform User Assessment

In block 302, the method may include performing an initial assessment ofthe user. A user, such as user 124, may be positioned in range of asensor, such as in front of the image capturing device 126 and/or sensor128, which may comprise an infrared transceiver. Display 136 may presenta representation of user 124 that may be a “mirror-image” or depict avirtual avatar, such as a user avatar, that moves to correspond withuser movement. Computer 102 may prompt the user to move into a certainregion relative to the image capturing device 126 and/or relative to theinfrared transceiver 128 so that the user is within frame and/or range.When properly positioned, system 100 may process movement of the user.Although the term “initial” has been utilized, this assessment may occureach time the user initiates system 100, performs certain movements,upon passage of time, or for any other reason. Thus, references toassessments herein are not limited to a single assessment.

a. Identify Sensory Locations

System 100 may process sensory data to identify user movement data. Inone embodiment, sensory locations on a user's body may be identified.With reference to FIG. 4, sensory locations 402 a-402 o may correspondto locations of interest on the user's 124 body (e.g., ankles, elbows,shoulders, etc.). For example, images of recorded video, such as fromcamera 126, may be utilized in an identification of the sensorylocations 402 a-402 o. For example, the user may stand a certaindistance, which may or may not be predefined, from the camera 126, andsystem 100 may process the images to identify the user 124 within thevideo, for example, using disparity mapping techniques. In an example,image capturing device 126 may be a stereo camera having two or morelenses that are spatially offset from one another and thatsimultaneously capture two or more images of the user. System 100 mayprocess the two or more images taken at a same time instant to generatea disparity map for determining a location of certain parts of theuser's body in each image (or at least some of the images) in the videousing a coordinate system (e.g., Cartesian coordinates). The disparitymap may indicate a difference between an image taken by each of theoffset lenses.

In a second example, one or more sensors may be located on or proximateto the user's 124 body at the sensory locations 402 a-402 o or the user124 may wear a suit having sensors situated at various locations. Yet,in other embodiments, sensor locations may be determined from othersensory devices, such as devices 138, 140 and/or 142. In this regard,sensors may be physical sensors located on a user's clothing, yet inother embodiments, sensor locations 402 a-402 o may be based uponidentification of relationships between two moving body parts. Forexample, sensor location 402 a may be determined by identifying motionsof user 124. In this regard, the overall shape or portion of a user'sbody may permit identification of certain body parts. Regardless ofwhether a camera, such as camera 126, is utilized and/or a physicalsensor located on the user 124, such as sensors within device(s) 138,140, 142 are utilized, the sensors may sense a current location of abody part and/or track movement of the body part.

In certain embodiments, a time stamp may be added to the data collected(such as collected part of block 302 in FIG. 3) indicating a specifictime when a body part was at a certain location. Sensor data may bereceived at computer 102 (or other device) via wireless or wiredtransmission. A computer, such as computer 102 and/or devices 138, 140,142, may process the time stamps to determine the locations of the bodyparts using a coordinate system (e.g., Cartesian coordinates) withineach (or at least some) of the images in the video. Data received fromcamera 126 may be corrected, modified, and/or combined with datareceived from one or more other devices 138, 140, and 142.

In a third example, system 100 may use infrared pattern recognition todetect user movement and locations of body parts of the user 124. Forexample, sensor 128 may include an infrared transceiver, which may bepart of camera 126, or another device, that may emit an infrared signalto illuminate the user's 124 body using infrared signals. The infraredtransceiver 128 may capture a reflection of the infrared signal from thebody of user 124. Based on the reflection, the system 100 may identify alocation of certain parts of the user's body using a coordinate system(e.g., Cartesian coordinates) at particular instances in time. Which andhow body parts are identified may be predetermined based on a type ortypes of exercise a user is requested to perform.

As part of a workout routine, system 100 may make an initial posturalassessment of the user 124 as part of the initial user assessment inblock 302 of FIG. 3. With reference to FIG. 5, system 100 may analyzefront and side images of a user 124 to determine a location of one ormore of a user's shoulders, upper back, lower back, hips, knees, andankles. On-body sensors and/or infrared techniques may also be used,either alone or in conjunction with camera 126, to determine thelocations of various body parts for the postural assessment. Forexample, system 100 may determine assessment lines 124 a-g and/orregions 502-512 to determine the locations of a various points on auser's body, such as, for example, ankles, knees, hips, upper back,lower back, and shoulders.

b. Identify Sensory Regions

In further embodiments, system 100 may identify sensory regions (see,e.g., block 302). In one embodiment, assessments lines 124 a-g may beutilized to divide the user's body into regions. For example, lines 124b-f may be horizontal axes. For example, a “shoulders” region 502 maycorrelate to a body portion having a lower boundary around the user'sshoulders (see line 124 b), region 504 may correlate to the body portionbetween the shoulders (line 124 b) and about half the distance to thehips (see line 124 c) and thus be an “upper back” region, and region 506may span the area between line 124 c to the hips (see line 124 d) tocomprise a “lower back region.” Similarly, region 508 may span the areabetween the “hips” (line 124 d) and the “knees” (see line 124 e), region510 may span between lines 124 e and 124 f and region 512 (see “ankles”)may have an upper boundary around line 124 f Regions 502-512 may befurther divided, such as into quadrants, such as by using axes 124 a and124 g. To aid in the identification of one or more sensory regions,system 100 may prompt the user to make one or more specific movements.For example, system 100 may prompt a user to move a specific body partor region (e.g., wave their right arm, or wave the left arm in aspecific pattern) to aid the system 100 (e.g., computer algorithmprocessing information received from the infrared transceiver 128) indetermining which body part or region is in a specific location within acoordinate system.

c. Categorize Locations or Regions

In certain embodiments, body parts or regions that are not proximate toeach other may nonetheless be categorized into the same movementcategory (see, e.g., block 302). For example, as shown in FIG. 5, the“upper back”, “hips”, and “ankles” regions 504, 508, 512 may becategorized as belonging to a “mobility” category. In anotherembodiment, the “lower back” and “knees” regions 506, 510 may becategorized as belonging to a “stability” category. The categorizationsare merely examples, and in other embodiments, a location or region maybelong to multiple categories. For example, a “center of gravity” regionmay be formed from regions 504 and 506. In one embodiment, a “center ofgravity” may comprise portions of regions 504 and 506. In anotherembodiment, a “center of moment” category may be provided, eitherindependently, or alternatively, as comprising a portion of at leastanother category. In one embodiment, a single location may be weightedin two or more categories, such as being 10% weighted in a “stability”category and 90% weighted in a “mobility” category.

System 100 may also process the image to determine a color of clothingof the user or other distinguishing features to differentiate the userfrom their surroundings. After processing, system 100 may identify alocation of multiple points on the user's body and track locations ofthose points, such as locations 402 in FIG. 4. System 100 may alsoprompt the user to answer questions to supplement the posturalassessment, such as, for example, age, weight, etc. Again, block 302 isoptional and is not required in accordance with various embodiments.

2. Providing Form

With reference again to FIG. 3, in block 304, various embodiments mayinclude demonstrating proper form for an exercise and prompting the userto perform the exercise. For example, after or in addition to theinitial postural assessment, the system 100 (such as with computer 102)may cause the display 136 to present a virtual trainer demonstrating anexercise to instruct the user on proper form and/or may present adepiction and/or an actual video of a real person demonstrating properform for an exercise. System 100 may then prompt the user to beginperforming the exercise.

With reference to FIG. 3, in block 306, various embodiments may includemonitoring form of a user performing the exercise. As seen in FIG. 6,system 100, such as through computer 102, may cause the display 136 topresent a virtual avatar 602 of the user. The virtual avatar 602 maymove in synchronism with the user 124. Also, the display 136 may presentvideo of the actual user, rather than avatar 602. System 100 may processone or more frames in the video to determine at least some of thesensory locations 402, or may receive data from sensors worn on-body bythe user. As shown in FIG. 6, sensory locations 402 may be displayed onthe virtual avatar.

For proper form during many exercise routines, a user may proceedthrough multiple positions during a repetition of an exercise. Certainaspects disclosed herein relate to defining one or more measurementpositions and/or desired locations for one or more sensory locations402. For example, a measurement position may refer to a particularrelationship between various body parts during a repetition. Forexample, a measurement position may indicate a desired location for auser's body part (e.g., desired location of user's left elbow) and mayindicate a desired relationship between multiple body parts (e.g., anglebetween a user's torso and thigh). For a movement or series of movements(such as an exercise routine), system 100 may define one or moremeasurement positions and/or desired locations for one or more of thesensory locations 402 for a measurement position. In variousimplementations, each repetition of an exercise can be broken down intoone or more measurement positions.

System 100, such as through computer 102, may process video or sensordata of a user performing an exercise to determine when a user's bodyhas reached a measurement position. For each measurement position,system 100 may compare the measured sensory locations to desired sensorylocations to monitor the user's form while performing the exercise. Forexample, frame 1 of FIG. 6 may correspond to a first measurementposition and frame 2 may correspond to a second measurement position.System 100 may determine a distance between sensory locations 402 c and402 d at each measurement position. Other relationships between sensorylocations may be specified (e.g., certain angle, certain position, etc.)

With reference again to FIG. 3, in block 308, various embodiments mayinclude calculating an energy expenditure estimate for the user.Calculations may be based on a type of the exercise, on the form of theuser and/or contribution value that correlates energy expenditure withform for a given exercise or drill. The energy expenditure estimate maybe, or comprise, for example, an estimate of calories burned by theuser. In certain embodiments, energy expenditure calculations comprisedeterminations relating to: effort, oxygen consumed, and/or oxygenkinetics of the user. During a workout session or upon its completion,the system 100 may inform the user of energy expended. In oneembodiment, system 100 may provide an indication of a quantity ofcalories they have burned. To provide a more accurate calories burnedestimate, system 100 may account for a user's form while performing anexercise as well as the type of exercise that was performed.

Form may impact energy expenditure during some exercises or drills andnot others. For example, performing a drill, such as an agility shuffle,with improper form may result in a lower or higher energy expenditurethan performing the same drill with proper form. In one embodimentmultiple aspects of a drill or exercise are monitored. The monitoringmay be performed with a camera, multiple cameras and/or other sensors.Each aspect of the exercise or drill may be assigned a form score and acontribution value that indicates the relevance of proper form for thataspect of the exercise or drill to energy expenditure. For example, asquat drill or exercise may have form scores for the level of backstraightness, depth of a squat and how well heels remain on the ground.A user may receive form scores of 0.3 for back straightness, 0.6 forsquat depth and 0.1 for how well her heels remain on the ground. In oneembodiment of the invention contribution values may be either one orzero. Of course, several other values may also be used. Returning to thesquat example, it may be determined by prior monitoring or calculationsthat back straightness and how well heels remain on the ground have norelevance to energy expenditure, while squat depth is relevant.Contribution values of zero, one and zero may be assigned for backstraightness, squat depth and how well heels remain on the ground,respectively. Form scores may be multiplied by contribution values whendetermining energy expenditure.

Further embodiments may utilize user attributes to more accuratelyidentify a number of calories burned by a user. Example user attributesmay be height, weight, age, etc. One or more sensors may determine theuser attributes, or the user may input the user attributes via aninterface to a computer, such as computer 102.

System 100 may use information from sensory locations 402 detected atmeasurement positions of an exercise in combination with one or moreknown values to obtain a more accurate determination of calories burned.In one embodiment, a known value may comprise or be part of a MetabolicEquivalent of Task (MET) table. A MET table, for example, may be definedfor a particular exercise (e.g., squat, lunge, etc.) and used todetermine how many calories a user burned during a workout. System 100may store or have access to multiple MET tables corresponding todifferent exercises (e.g., squat, lunge, jumping rope, push up, running,etc.). System 100 may process data from the video and/or sensors todetermine a number of repetitions of an exercise that a user hasperformed or duration of an exercise, and may estimate a number ofcalories burned by the user based on the repetitions and/or durationinformation and the one or more known values, such as may be obtainedfrom MET tables.

MET tables, however, are statistical averages and are not as accurate asthey could be. Thus, conventional calorie measurement systems that relyon MET tables merely provide a user with a rough estimate of how manycalories they burned during a workout. Although embodiments of thisdisclosure may utilize one or more values from a MET table, aspects ofthis disclosure are not limited by the deficiencies of priormeasurements systems. For example, in one embodiment the user's form maybe accounted for. System 100 may apply a scaling factor to a caloriesburned estimate based on detected sensory location information. Thescaling factor may reflect how well a user has performed an exercise andin certain embodiments may consider attributes of the user. For example,the scaling factor may be a function of one or more of the sensorylocation information, a duration during which the user performed anexercise, information reported by the user (e.g., age, weight), a user'sheart rate taken by a heart rate monitor, a pressure measurement, and/orother data. A pressure measurement may be obtained from pressure sensor140 located in a shoe, for example, to determine how much force a userexerts during movement. For example, a user may be holding a weight ineach hand and the pressure sensor 140 may monitor pressure at the shoe.The pressure sensor 140 may also indicate how quickly a user changesdirection (e.g., how hard a user made a cut) or how much power wasexerted when jumping.

To determine the scaling factor, system 100 may monitor forrelationships between one or more body parts at one or more measurementpositions during a repetition of an exercise. Modifications to theserelationships may make an exercise easier or harder to perform. Thescaling factor may consider factors indicative of whether a user ismaking the exercise more or less difficult to complete, and may adjust acalories burned estimate accordingly. In a squat, for example,relationships may be defined for a first angle between a user's torsoand thighs, and a second angle between a user's thighs and shin whileperforming the squat. System 100 may process sensory locationinformation to measure the first and second angle of the user over timefor comparison with the desired first and second angle.

In an example, with reference to FIGS. 7A-B, a virtual avatar 702 of auser is displayed performing a squat. Virtual avatar 702 is depicted asa stick figure, and proper technique for an exercise is shown as ashaded region 704. At the lowest part of the squat (for example, asshown in FIG. 7A), the desired form may specify a relationship between auser's thigh and shin, between a user's back and arms, and/or any othertwo parts or locations the user. In one embodiment, the desired form mayspecify a first predetermined angle between a location or part. Forexample, a user's upper leg and lower leg, and/or a second predeterminedangle between a user's back and arms. System 100 may process the sensorylocation information to compare the user's form to the desired form. Forexample, system 100 may process the sensory location information todetermine an angle between the user's thigh and shin, and an anglebetween the user's back and arms when performing a squat.

System 100 may define thresholds for the relationships between variousbody parts for adjusting the scaling factor. The thresholds may permitthe user's form to differ by a certain amount from the desired form. Fora preferred threshold, system 100 may determine that the user has goodform that does not require any adjustment of the scaling factor (e.g.,less than a 5% difference between angle between the user's upper leg andlower leg and desired angle). For an acceptable threshold, the system100 may nominally adjust the scaling factor upward or downward toreflect increased or reduced effort by the user (e.g., 5-15% differencebetween angle between the user's upper leg and lower leg and desiredangle). For an unacceptable threshold, the system 100 may determine thatthe user's form has reduced the amount of effort to perform the exerciseand may downwardly adjust the scaling factor (e.g., greater than a 15%difference between angle between the user's upper leg and lower leg anddesired angle).

System 100 may also adjust the scaling factor based on omissions oradditions a user makes when performing an exercise. For example, a usermay not be doing an arm movement in an exercise that requires movementof both arms and legs. Also, if the user is performing an additionalmovement beyond what is specified for an exercise, the system 100 mayadjust the scaling factor to increase the calorie estimate.

Upon determining the scaling factor, the system 100 may determine anamount of calories burned as a function of the scaling factor(s) and thecalorie estimate. The function may be a multiplication of the calorieestimate by the scaling factor, or via other relationships. For example,the scaling factor may be adjustments to a number of variables in amathematical equation for adjusting calories burned by one or more ofmultiplication, addition, and subtraction. In further embodiments,system 100 may cease determinations relating to caloric expenditure ifthe user deviates from a threshold. For example, a user may beinterrupted during a workout routine and either forget or be toodistracted to “pause” the determination, thus, certain embodiments maycease determining caloric expenditure upon detecting that a user is notperforming an exercise. Further embodiments may cease or otherwise alterdeterminations of caloric expenditure if one or more variationthresholds are exceeded, such as for example, if a user isover-extending or under-extending a body region or part. In certainembodiments, if a user's movements are prone to cause injury,measurements and/or determinations relating to caloric expenditure maybe stopped. In one implementation, system 100 may provide cues and/orinstructions to correct the user's deficiencies or incorrect movements.

The following provides an example equation for calculating an amount ofcalories burned by a user during a workout.Calories burned=BMR*(Activity modifier)*(Completenessmodifier).  (Equation (1):

In equation (1), BMR is an acronym for Basal Metabolic Rate. The system100 may calculate the BMR using the Mifflin-St. Jeor Equation,BMR=(10*w)+(6.25*h)−(5.0*a)+(5 for men, −161 for women), where “*” isthe multiplication symbol, “w”=weight in kilograms, “h”=height incentimeters, “a”=age in years. The system 100 may also use theHarris-Benedict equation instead of or, in addition to, the Mifflin-St.Jeor Equation.

The activity modifier may be an adjustment corresponding to a type ofexercise being performed by a user. The activity modifier may be largerfor more strenuous exercises, and smaller for less strenuous. System 100may store a file containing activity modifiers, where each activitymodifier may have a value for a particular exercise type. Two or moreexercises may have activity modifiers with a same value, or certainexercise may have a unique value for the activity modifier. The activitymodifier may have a default value. In one example embodiment, thedefault value may be 0.1. In a second embodiment, the default value maybe 1.0. The default value may be any value, including 0.0. System 100may update the default value to correspond to the activity modifier foran exercise currently being performed by the user. Over a duration ofthe workout, system 100 may use different ones of the activity modifiersto calculate calories burned using equation (1) corresponding todifferent exercises the user is prompted to perform. One or more factorsmay contribute to the activity modifier and/or adjustment of themodifier. Examples include, but are not limited to: pace, type ofexercise, duration, and combinations thereof. Further, activitymodifiers and/or variation of activity modifiers may be determined frompredetermined values (such as a value assigned to an exercise ormovement that a user is prompted to perform), the user's performance,information from a MET table on a particular exercise, and combinationsthereof.

The completeness modifier may be used for adjusting the BMR based on howwell a user's form corresponds to a desired form when performing anexercise. In an example, the completeness modifier may indicate whatpercentage of full movement was achieved for each repetition whenperforming an exercise (e.g., determine a percentage of a measured anglebetween the user's torso and thighs for a particular repetition of anexercise relative to a desired angle), or may be an average of thepercentage of full movement for a predetermined number of repetitions(e.g., last three exercises, last five exercises, all exercises, etc.).The completeness modifier may have a default value. In one exampleembodiment, the default value may be 0.1. In a second embodiment, thedefault value may be 1.0. The default value may be any value, including0.0. System 100 may update the completeness modifier over time based onhow well the user's form conforms to a desired form. One or more factorsmay contribute to the activity modifier and/or adjustment of themodifier. Examples include, but are not limited to: pace, type ofexercise, duration, and combinations thereof. Further, activitymodifiers and/or variation of activity modifiers may be determined frompredetermined values (such as a value assigned to an exercise ormovement that a user is prompted to perform), the user's performance,and combinations thereof.

Equation (2), provided below, may be utilized in further embodiments.Calories burned=BMR*(Activity modifier)*(Completenessmodifier)*(Multiply Modifier)+(Addition Modifier)  Equation (2):

Values for BMR, Activity Modifier, and/or Completeness Modifier ofEquation (2) may be determined in accordance with one or moreembodiments described above in reference to Equation (1). In oneembodiment, the value of the Multiply Modifier may be defined for eachtype of exercise. In one example embodiment, the default value may be0.1. In a second embodiment, the default value may be 1.0. The defaultvalue may be any value, including 0.0. System 100 may update theMultiply Modifier during a workout to correspond to a type of exercisethe user is prompted to perform. In certain embodiments, the ActivityModifier may be obtained (either partially or entirely) from empiricaldata.

In certain embodiments, the value of the Addition Modifier may bedefined for each type of exercise. In one example embodiment, thedefault value may be 0.1. In a second embodiment, the default value maybe 1.0. The default value may be any value, including 0.0. System 100may update the Addition Modifier during a workout to correspond to atype of exercise the user is prompted to perform. In certainembodiments, the Activity Modifier may be obtained (either partially orentirely) from empirical data.

System 100 may calculate the calories burned over a duration of aworkout, which may incorporate the utilization of equations (1) or (2).System 100 may cause the display 136 to display a running total ofcalories burned. In certain embodiments, the total may be determined forone or more completed repetitions and one or more completed sets of eachexercise. System 100 may also calculate and cause display of caloriesburned by type of exercise performed. Other information such as, forexample, peak/minimum/average calorie burning rate by workout, byrepetition, by set, or by exercise type may also be calculated anddisplayed. System 100 may periodically determine an amount of caloriesburned by the user while exercising using equation (1). System 100 mayindicate a current amount of calories burned that is continually updatedover a workout (e.g., a running total), or may update the caloriesburned amount at predetermined times (e.g., user completes a set of afirst type of exercise and begins a set of second type of exercise, atthe end of the workout session, etc.). System 100 may also inform theuser how many calories were burned during each repetition as well as ineach set of an exercise.

One or more of the inputs and/or variables used in the determination ofcaloric expenditure (such as with equation (1)) may remain the sameregardless of the type of exercise being performed by the user, yetothers may vary. For example, the BMR may be the same over the entireworkout as a user's weight, height, and age do not change appreciablyover the course of a workout. Further, one or more of the Activitymodifier, Completeness modifier, Multiply Modifier, and AdditionModifier may vary over the workout. The values (and/or variation) of thevalues may depend on the type exercise currently being performed by theuser.

The Completeness modifier may vary from repetition to repetition. Asnoted above, system 100 may generate the Completeness modifier based onmonitoring a user's form while they perform an exercise. Generally, anexercise includes a sequence of motions to perform one repetition, and auser typically performs a set that includes two or more repetitions. Auser's form may vary from repetition to repetition, and so may theCompleteness modifier.

System 100 may determine calories burned using equation (1) based on aCompleteness modifier that varies from repetition to repetition, orbased on a filtered version of the Completeness modifier. To filter theCompleteness modifier, the system 100 may, for example, determine aCompleteness modifier for one or more repetitions, may average some orall of the Completeness modifiers, and may use the average in equation(1). Also, system 100 may generate the Completeness modifier as aweighted average, where Completeness modifiers of some repetitions maybe given greater weight than others. For example, system 100 may apply adecaying function where more recent Completeness modifiers are weightedmore heavily than less recent when generating an average.

System 100 may also allow a user to make desired movements, andcalculate an amount of calories burned for such movement. In oneembodiment, all detected movements may be utilized in calculations. Yetin other embodiments, only certain (e.g., system supported and/or thoseprompted to be performed) movements may be considered. System 100 mayprocess data from image capturing device 126 and/or from various sensorsto attempt to classify a user's movement. For example, system 100 maycompare the user's movement to other known movements for which a METtable has been defined. If a user's movement corresponds to a knownmovement for which a MET table has been defined, then system 100 may usethe identified MET table for calculating an amount of calories burned.

If the user's movement does not match an exercise defined by a METtable, the system 100 may identify one or more exercises that includemovements similar to the movement being performed by the user. Forexample, system 100 may determine that the user's lower body movessimilar to a squat and upper body moves similar to a pushup. System 100may calculate the number of calories the user would burn using theidentified MET tables as if the users were doing a squat, and as if theywere doing a pushup, as approximations for the amount of calories burnedby the user. In further embodiments, a new entry may be created. In thisregard, certain embodiments may permit the entry and lateridentification of new movements and/or exercises. In certainembodiments, the user may provide inputs regarding an approximatecaloric expenditure for an unidentified movement/exercise. Yet in otherembodiments, system 100 may calculate caloric expenditure, such as fromone or more sensors as discussed herein. In still yet furtherembodiments, system 100 may utilize one or more sensor readings as wellas an input from a user (and/or third-party) in determining attributes,such as caloric expenditure, for previously unknown movements orexercises. Examples of estimating caloric expenditure without METtables, may include but are not limited to, determining changes inpotential energy. Examples of using changes in potential energy areprovided in the next section.

System 100 may be configured to transmit calories burned estimates to asocial networking website. The users may be ranked based on their totalnumber of calories burned for a desired time interval (e.g., rank byday, week, month, year, etc.). With reference again to FIG. 3, themethod may end or may return to any of the preceding blocks.

i. Energy Expenditure Estimate Based on Changes in Potential Energy

System 100 may also calculate an energy expenditure estimate of a userfor physical activities not defined by a MET table. For example, system100 may calculate an amount of calories burned by a user performing anydesired combination of movements. During a workout, a user may beexposed to their own body weight and gravity. A location of a user'scenter of mass, or of a center of mass of a particular body part, may beutilized in estimating an amount of calories burned by the userperforming an athletic activity.

FIG. 8 illustrates an example flow diagram of a method for calculatingan energy expenditure estimate for a user while performing an athleticactivity based on monitoring changes in potential energy, in accordancewith example embodiments. The method may be implemented by a computer,such as, for example, computer 102, device 138, 140 and/or 142 as wellas other apparatuses. The blocks shown in FIG. 8 may be rearranged, someblocks may be removed, additional blocks may be added, each block may berepeated one or more times, and the flow diagram may be repeated one ormore times. The flow diagram may begin at block 802.

In block 802, various embodiments may involve processing data capturedof a user performing an athletic activity over a time interval. In anexample, system 100 may prompt a user to perform ten repetitions of alunge and may process data captured of the user performing the lunge.The data may be video captured by the camera 126 or may be captured bythe infrared transceiver 128, and/or by the other device sensors 138,140, and 142.

In block 804, various embodiments may involve determining a location ofa center of mass of a body part, body region, or of an entire body ofthe user at a first time instant and at a second time instant within thetime interval. Yet in other embodiments, a center of movement may beutilized. For simplicity purposes, however, a center of mass will bediscussed. In an example, system 100 may instruct the user to placesensors at locations of corresponding to a center of mass for one ormore body parts of the user. With reference to FIG. 9, one or more ofcenter of mass locations may be at example locations 904A-D and 906, orat other locations on the user's body. Any number of locations may bemonitored. At least one sensor may wirelessly transmit sensor dataindicating a time and a location of the sensor (or location of a bodypart as detected by the sensor). A location may be coordinates in acoordinate system (e.g., Cartesian coordinate system) and may beassociated with a time stamp indicating when the sensor was at aparticular coordinate. In certain embodiments, system 100 may processthe sensor data to periodically determine locations 904A-D and 906. Forexample, system 100 may receive sensor data, such as from device sensors138, 140 and/or 142. Computer 102 (or another component of system 100)may process data as part of determining locations (such as locations904A-D and 906). In one embodiment, data may be processed on a routineongoing-basis, such as four times per second. In another example,computer 102 (or another component of system 100) may process data fromimage capturing device 126 to determine locations 904A-D and/or 906.

In block 806, various embodiments may involve identifying a change inthe location of the center of mass from the first time instant to asecond time instant. As discussed above, system 100 may determinelocations 904A-D and 906 at one time and at a subsequent time. Forexample and with reference to FIGS. 10A-B, a user is shown performing alunge. FIG. 10A corresponds to a first time instant and FIG. 10Bcorresponds to a second time instant. In FIG. 10A, a location 906 of auser's center of mass is at a height “h1” (designated by 908A) off ofthe ground. In FIG. 10B, a location 906 of a user's center of mass is ata height “h2” (designated by 908A) off of the ground. One or morecomponents of system 100 may determine a difference between height “h1”and “h2” to determine a change in a location 906 of the center of mass.System 100 may also calculate changes to locations 904A-D of centers ofmass for other body parts, or changes to other locations of body partsor body regions of the user. System 100 may also process video of a usertaken from different angles, as shown in FIG. 11, to determine locations904A-D and 906. For example, system 100 may determine height “h1” forlocation 906 in a perspective view and height “h2” for location 906 in afront view of the user. System 100 may average the different heightmeasurements, or may use one or the other.

With reference again to FIG. 8, in block 808, various embodiments maycalculate an energy expenditure estimate for the user due to the change.In an example, the physics concept of potential energy may be used toestimate the amount of work done by the user, and to calculate caloriesburned based on work.

In an example, one or more components of system 100 may determinechanges of a location 906 from one time instant to another to determinean amount of work performed by the user. Potential Energy (PE)=m*g*h,where m=mass of the user (or body part), g=the acceleration due togravity, and h=height above ground. Work (W)=−ΔPE, where Δ is representsa change in potential energy. Substituting m*g*h, Work (W)=−m*g*Δh.Based on the above example in FIGS. 10A-B, W=−m*g*(h1−h2). System 100may determine an amount of calories burned as a function of workmultiplied by physiology of human efficiency. System 100 may determinethe amount of calories burned based on the amount of work and aphysiology of human efficiency (PHE) scaling factor. The system 100 maydetermine the PHE scaling factor as a function of one or more of theuser's heart rate, pressure sensor data, and other information input bythe user (e.g., age, weight, etc.)

System 100 may keep and/or transmit a running total of calories burnedbetween subsequent time instants and inform the user of a total amountof calories burned up to that point in an exercise session. For example,system 100 may determine a height h of location 906 at a certainfrequency (e.g., 2 times per second), and may calculate calories burnedbased on a difference in calories burned between each determination ofheight h. The system 100 may also track a total number of caloriesburned over a predetermined time range covering one or more workouts. Atime range may include a week, month, year, cumulative time since a userbegan working out, or other defined metrics. One or metrics may comprisedefault values, predefined values, user-selectable values, and/oruser-defined values. For example, system 100 may inform the user of howmany calories they have burned during a specified time period, such as aday, week, month, and/or year. System 100 may also maintain data onaverage number of calories burned per workout, average number ofcalories burned based on a type of workout, a greatest number ofcalories burned during a single workout or during a predetermined timeinterval (e.g., month where highest amount of calories were burned), orother types of data.

In another example, system 100 may determine calories burned by movementof a particular body part or by a collection of body parts. Forinstance, a user may desire to know how many calories were burned bymovement of their right leg. Using the above relationship between workand potential energy, and with reference again to FIG. 9, system 100 maymonitor changes in the location 904A of the center of mass of the user'sright leg (e.g., height 908B) from one time instant to a different timeinstant to calculate work. System 100 may estimate the mass of theuser's right leg based on the user's weight and proportions. System 100may then determine an amount of calories burned as a function of workmultiplied by physiology of human efficiency, as described above. Duringan exercise session, system 100 may display, such as through display136, a running total of calories burned attributable to movement of theuser's right leg. System 100 may similarly determine calories burnedbased on locations 904B-D for the other limbs of the user. During anexercise session, system 100 may display a running total of caloriesburned by a user's entire body, as well by each limb.

System 100 may also permit a user to review an exercise session todetermine how many calories were burned at certain times. For example,an exercise may involve performing repetitive motions (e.g., pushups).System 100 may identify each repetition within a set (e.g., each pushupwithin a set of 10), as well as a number of calories burned during eachrepetition. Over a set, one or more components of system 100 mayidentify the repetition where the user burned a highest number ofcalories as well as a lowest number of calories. In further embodiments,system 100 may estimate an average number of calories. These are merelyexemplary statistics and those skilled in the art will readilyappreciate that other analysis may be conducted without departing fromthe scope of this disclosure.

If an exercise session involves different types of exercises, system 100may rank the exercise types based on the amount of calories burned bytype. For example, an exercise session may involve 3 different types ofexercises (e.g., pushups, sit-ups, squats). After completing theexercise session, system 100 may determine how many calories were burnedby each exercise type (e.g., 10 calories for pushups, 13 calories forsit-ups, and 18 calories for squats), and rank the exercise types basedon the number of calories burned (e.g., first squats, second sit-ups,third pushups). In further embodiments, energy expenditure (e.g., aquantity of calories burned) may be ranked as percentage over an idealvalue or range for an exercise or routine. For example, if perfectlyperforming an exercise would burn about 100 calories, a first user whoburned 90 calories may be assigned a better ranking than second user whoonly burned 85 for the same exercise. The users could have differentideal values or ranges, thus the determinations may utilize thepercentage of the detected and/or estimated values as a percentage forthat user's ideal value. In further embodiments, a user who is closer to100% of their ideal value may be ranked higher than users who have over100% of the ideal quantity of calories burned. In this regard, a userwho expends more energy than estimated or calculated for an activity(e.g., exercise) may indicate improper movements, inefficiency,increased likelihood of injury, and/or combinations thereof. In certainimplementations, the method of FIG. 8 may then end, or may return to anyof the preceding blocks and/or other processes.

System 100 may also determine calories expended from pre-recordedvideos. For example, a user may upload video of a professionalbasketball player dunking a basketball to system 100. One or morecomponents of system 100 may process the video to determine locations ofa center of mass of the player, or of particular body parts, at variouspoints in time, and determine the amount of calories expended during thephysical activity (e.g., by the player during the dunk) using thework-based calorie determination, described above.

Conclusion

Providing an activity environment having one or more of the featuresdescribed herein may provide a user with an immersive experience thatwill encourage and motivate the user to engage in athletic activitiesand improve his or her fitness. Users may further communicate throughsocial communities and challenge one another to reach various levels offitness, and to view their fitness level and activity.

Aspects of the embodiments have been described in terms of illustrativeembodiments thereof. Numerous other embodiments, modifications andvariations within the scope and spirit of the appended claims will occurto persons of ordinary skill in the art from a review of thisdisclosure. For example, one of ordinary skill in the art willappreciate that the steps illustrated in the illustrative figures may beperformed in other than the recited order, and that one or more stepsillustrated may be optional in accordance with aspects of theembodiments.

What is claimed is:
 1. A computer-implemented method comprising:prompting, at a first location, a user to perform an exercise comprisingmultiple repetitions; determining a location of a computer device on abody of the user, from data received from a sensor integrated into anitem of apparel worn by the user; calculating, by the computer device,an angle between a first body part and a second body part of the user,based on the determined location; monitoring, at the first location,with at least a first sensor of the computer device, a form of the userwhile performing the exercise, based on the calculated angle, whereinthe form of the user comprises at least two form scores and at least twocompleteness modifiers corresponding to at least two parts of theexercise, wherein the at least two completeness modifiers adjust the atleast two form scores based on how well the user's form corresponds to adesired form, and wherein each of the at least two completenessmodifiers is a weighted average comprising a decaying function thatweights more heavily recent repetitions of the exercise; andcalculating, by a processor, an energy expenditure estimate for the userperforming the exercise based on a type of the exercise, the form of theuser, and at least two contribution values that correlate the at leasttwo form scores with energy expenditure.
 2. The method of claim 1,wherein the calculating of the energy expenditure estimate is determinedutilizing a metabolic equivalent of task (MET) table for the type of theexercise.
 3. The method of claim 1, wherein the energy expenditureestimate is based on a basal metabolic rate of the user.
 4. The methodof claim 1, wherein the monitoring of the form is based on processing ofvideo of the user performing the exercise.
 5. The method of claim 1,wherein the monitoring of the form comprises: processing of an infraredreflection of the user.
 6. The method of claim 1, wherein the monitoringof the form is based on identifying locations of a first body part ofthe user with respect to at least a second body part of the user, atdifferent times.
 7. The method of claim 6, wherein at least a portion ofthe body part locations are determined based on one or more of data froma plurality of sensors positioned on the body of the user, processingvideo of the user, and processing infrared reflections of the user. 8.The method of claim 1, wherein the calculating of the energy expenditureestimate comprises increasing or decreasing the estimate based on theform and the at least two contribution values.